could you add a check list or some ideas what may need to be added here?

could you add a check list or some ideas what may need to be added here?

Will do. Right now it needs someone who tests it on a non trivial project, that has real unit tests. That will likely be myself. And I'm blocked by time right now.

I'll play with it today on sidekiq's suite.

I'll play with it today on sidekiq's suite.

I tried with sidekiq for maybe 10min back one month or so. But could not isolate a test / subject set that is a "real unit" test for testing the rspec integration.

By that I mean that most of the tests (and maybe even the global setup, I do not recall precicely) require redis / and or cause side effects.

You should start with -j1 to limit concurrency. I had some local commits that made the sidekiq test suite more accessible for mutant, but I lost them in the chaos that is my current live (sick for 4+ weeks).

Conceptually I think that sidekiq might not be the best project to verify the mutant-minitest integration. Since its domain is mostly managing nondeterministic things (threads, network, ...).

Not that I think sidekiq and mutant "cannot" play well together, but I think its at too big step. It would be easier to start with a smaller lib that does not have IO etc to validate the integration is correct, before using the integration on something bigger and running into false positives that burn time.

How would I try this out?

How would I try this out?

No time at all sorry. Someone with a less "global state hitting" minitest project should try.

I mean just, how would I invoke this from the command line? minitest in place of rspec?

I mean just, how would I invoke this from the command line? minitest in place of rspec?

--use minitest yes. And use this branch.

I gave it a try on a Rails project, and it executed part-way but then had an error:

RAILS_ENV=test bundle exec mutant -r ./config/environment -I test --use minitest Agency

Output:

Mutant configuration:
Matcher:         #<Mutant::Matcher::Config match_expressions: [Agency]>
Integration:     Mutant::Integration::Minitest
Expect Coverage: 100.00%
Jobs:            8
Includes:        ["test"]
Requires:        ["./config/environment"]
Subjects:        14
Mutations:       729
Kills:           0
Alive:           0
Runtime:         0.00s
Killtime:        0.00s
Overhead:        Inf%
Coverage:        100.00%
Expected:        100.00%
Active subjects: 0
/Users/robb/.rbenv/versions/2.2.3/lib/ruby/gems/2.2.0/bundler/gems/mutant-36269e744bcd/lib/mutant/integration/minitest.rb:60:in `expression_syntax': undefined method `cover_expression' for BusinessesControllerTest:Class (NoMethodError)
    from /Users/robb/.rbenv/versions/2.2.3/lib/ruby/gems/2.2.0/bundler/gems/mutant-36269e744bcd/lib/mutant/integration/minitest.rb:138:in `construct_test'
    from /Users/robb/.rbenv/versions/2.2.3/lib/ruby/gems/2.2.0/bundler/gems/mutant-36269e744bcd/lib/mutant/integration/minitest.rb:123:in `block in all_tests_index'
    from /Users/robb/.rbenv/versions/2.2.3/lib/ruby/gems/2.2.0/bundler/gems/mutant-36269e744bcd/lib/mutant/integration/minitest.rb:122:in `each'
    from /Users/robb/.rbenv/versions/2.2.3/lib/ruby/gems/2.2.0/bundler/gems/mutant-36269e744bcd/lib/mutant/integration/minitest.rb:122:in `each_with_object'
    from /Users/robb/.rbenv/versions/2.2.3/lib/ruby/gems/2.2.0/bundler/gems/mutant-36269e744bcd/lib/mutant/integration/minitest.rb:122:in `all_tests_index'
    from /Users/robb/.rbenv/versions/2.2.3/lib/ruby/gems/2.2.0/gems/memoizable-0.4.2/lib/memoizable/method_builder.rb:117:in `call'

(etc.)

Likely this is triggered from missing autoload triggers. Typical rails symptom. Try to explicitly require the file that defines that method. Or ideally test on a non rails project.

It seems that Mutant expects the missing method #cover_expression. (?) I can't find any references to that online. What should be providing that?

        # Cover expression syntaxes
        #
        # @return [Array<String>]
        #
        # @api private
        def expression_syntax
          klass.cover_expression
        end

https://github.com/mbj/mutant/blob/feature/minitest-integration/lib/mutant/integration/minitest.rb#L54-L61

@dogweather Ahh, I remember see the discussion in #330.

Depends on #508 from software rusting.

@dkubb Ready for review. Includes #508 so should not be merged before.

hi @mbj! I know you haven't looked at this in a while, but would you mind if I took over this work? I've been able to get this running on my local apps, the only real work I did was add .cover_expression to all test cases, which I could programmatically infer by the class name. This definitely worked for me and I was able to actually get mutations to run on my app, which is a fairly large Rails app using many engines and hundreds of models.

https://github.com/tubbo/mutant/tree/feature/minitest-integration

hi @mbj! I know you haven't looked at this in a while, but would you mind if I took over this work?

I'd be super happy if you did so.

Feel free to open a fresh PR. I'd close this one and reference it.

So the news is I've got a sponsor for moving this feature into a released version:

• Initially we'll have to keep the cover_expression "hack".
• We'll use auom as the corpus test, which will get a separate minitest suite. This is also nice to compare mutant side by side on both rspec and minitest (I think minitest will be faster).

Prelimiary results are good, AUOM is fully covered. I do want to polish the cover expression mechanism a bit more, and possibly add a bit better "selection reporting" before release.

This is on how an mutant minitest integration may look like: mbj/auom#19

There is a pre-release on rubygems. v0.0.2: https://rubygems.org/gems/mutant-minitest

The "official" release is pending, will be paired with better docs. Version will than jump to the mutant version, which is 0.8.19 as time of writing this.

I'd apprechiate the following:

• People trying it on a real code base, follow the AUOM integration here: https://github.com/mbj/auom/blob/master/test/unit/auom_test.rb#L3-L10
• Feedback (especially crashes)
• Possible getting me access to a commercially relevant minitest code base. So I can see it in action and iron out some bugs.